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Abstract 


This  paper  addresses  the  issues  of  experience  and  professional  certification,  and 
explores  the  following  questions:  Can  experience  be  accelerated  to  bolster 
certification  effects  across  the  range  of  professions?  Are  there  any  innovative 
methodologies  that  can  appreciably  accelerate  experience  and  shrink  the  time  it 
takes  to  achieve  it?  If  so,  many  professionals,  including  Defense  Acquisition 
Workforce  personnel,  could  be  the  beneficiaries  since  their  certification  levels  rely 
heavily  on  experience  (in  addition  to  education  and  training).  The  Defense 
Acquisition  Workforce  Improvement  Act  of  1990  became  law  21  years  ago,  but 
experience  shortfalls  are  still  surfacing.  If  left  alone,  these  experience  shortcomings 
could  result  in  acquisition  limitations  and  delay  the  fielding  of  essential  systems  that 
warfighters  need.  It  is  time  to  take  another  look  at  the  experience  variables  that  are 
extremely  important  in  the  acquisition  workplace  performance  equation.  What 
matters  and  what  doesn’t? 

Introduction 

In  any  business,  trade,  or  profession,  experience  matters,  especially  when  our  lives 
depend  on  it.  Not  surprisingly,  the  public  tends  to  look  at  experience  as  an  absolute 
necessity  when  personal  safety  is  paramount.  Professions  like  the  medical,  transportation, 
and  construction  industries  rely  heavily  on  experience.  They  take  considerable  time  to 
qualify  their  respective  corps  through  various  experience  incubators  like  internships, 
fellowships,  apprentices,  etc. — all  on-the-job  means,  and  for  obvious  reasons.  They  learn 
by  “doing.”  Without  doing,  these  personnel  may  face  challenges  later  that  they  cannot 
easily  overcome  when  “know-how”  matters  the  most.  As  a  result,  and  for  practical  reasons, 
many  of  these  professions  use  quantitative  measures  such  as  “hours”  or  “years.”  They 
serve  as  experience  markers.  It  not  only  gives  these  trades  more  confidence — it  also  gives 
the  public  more  confidence.  After  all,  assured  and  demonstrated  competencies  are  a  vital 
necessity  since  an  experience  failing  could  lead  to  life-threatening  consequences.  No  one 
wants  to  rely  solely  on  fatal  experiences  to  avoid  future  catastrophes. 

Many  of  these  same  professions  are  also  backed  up  by  licensing  boards  focused  on 
maintaining  minimum  standards.  For  example,  burgeoning  surgeons  spend  many  years 
practicing  their  craft  under  the  watchful  eye  of  experienced  surgeons  before  they  ever  get 
sanctioned  as  qualified  surgeons.  Entry-level  military  and  commercial  airline  pilots  must 
earn  a  minimum  number  of  successful  flight  hours  under  a  wide  range  of  operating 
conditions  before  they  are  allowed  to  climb  into  the  left  seat  (from  the  right  seat)  as  qualified 
pilots-in-command.  To  make  sure  they  do  not  become  an  electrical  danger  to  themselves  or 
anyone  else,  apprentice  electricians  require  a  minimum  number  of  years  as  apprentices 
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under  the  close  supervision  of  a  senior  lineman  before  they  go  solo  installing  or  repairing 
electrical  lines.  In  all  these  cases,  fundamentals  like  educational  achievement,  aptitude, 
previous  job  performance,  and  so  forth  serve  as  initial  career  screening  mechanisms. 
However,  the  existence  of  a  certification  or  qualification  component  tightly  connected  to 
experience  levels  seems  to  be  a  distinguishing  characteristic  that  makes  these  particular 
professions  different  enough  from  those  without  one.  Certification  also  serves  as  the  basis 
for  expected  outcomes.  But,  can  experience  be  accelerated  to  bolster  certification  effects 
across  the  range  of  professions?  Are  there  any  innovative  methodologies  that  can 
appreciably  accelerate  experience  and  shrink  the  time  it  takes  to  achieve  it?  If  so,  many 
professionals,  including  Defense  Acquisition  Workforce  (DAW)  personnel,  could  be  the 
beneficiaries  since  their  certification  levels  rely  heavily  on  experience  (in  addition  to 
education  and  training).  Twenty-one  years  after  the  Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA)  of  1990  became  law,  experience  shortfalls  are  still  surfacing.  If 
left  alone,  these  experience  shortcomings  could  result  in  acquisition  limitations  and  delay 
the  fielding  of  essential  systems  that  warfighters  need.  It  is  time  to  take  another  look  at  the 
experience  variables  that  are  extremely  important  in  the  acquisition  workplace  performance 
equation.  What  matters  and  what  doesn’t? 

Methodology 

This  investigative  effort  used  a  phenomenographical  methodology  (i.e.,  aggregate 
views  drawn  from  personnel  experiences)  by  surveying  a  wide  range  of  acquisition 
professionals  (e.g.,  program  managers,  systems  engineers,  logisticians,  contract  specialists, 
and  budget,  cost  estimators,  and  financial  managers)  in  various  product  lines  (e.g.,  ships, 
tanks,  aircraft,  satellites,  munitions,  information,  warfare,  etc.)  and  services  (e.g.,  IT, 
research,  security,  etc.).  This  investigation  sought  their  views  on  experience  catalysts. 

More  specifically,  what  mattered  more  to  them  than  others,  what  didn’t  matter,  and  why? 

The  answers  to  these  key  questions  would  confirm  key  experience  solutions  that  could  help 
fortify  the  capabilities  of  the  professional  acquisition  corps  and  combat  the  uncertain  and 
sometimes  turbulent  programmatic  challenges  that  lay  ahead. 

The  survey  separated  experience  catalysts  (EC)  into  three  tiers:  foundational  (Tier  1 
[T1  ]),  enhancers  (Tier  2  [T2]),  and  accelerators  (Tier  3  [T3]).  Decomposing  them  into  these 
tiers  would  afford  a  more  definitive  analysis  later.  This  partitioning  might  also  lend  itself  to  a 
greater  understanding  of  experience  gateways  as  well  as  the  prevailing  obstacles  (real  or 
artificial)  that  could  be  interfering  (in  the  form  of  barriers)  with  experience  gains  along  the 
acquisition  “experience  building”  pathway.  Mathematically,  the  total  sum  of  these  factors 
would  look  something  like  the  following: 


n 

EC  =  X  (Tier  1,  +  Tier  2,  +  Tier  3,)  -  Barriers, 
/=  1 


(1) 


Findings 


1,414  defense  acquisition  personnel  (1,236  government,  152  military,  and  26  support 
contractors)  responded  to  this  survey.  The  results  reinforced  both  the  importance  and 
influence  of  a  wide  range  of  experience  catalysts  operating  inside  and  outside  of  the 
workplace.  The  data  exposed  a  few  that  were  not  operating  at  expected  levels.  The  results 
also  generated  several  “ahas.” 

1st  Tier:  Experience  Foundations 
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Many  professions  rely  on  sturdy  and  enduring  academic  foundations.  The 
acquisition  profession  is  no  different.  Depending  on  the  specific  functional  area(s)  a 
member  of  the  DAW  pursues,  these  academic  foundations  tend  to  serve  as  formal  learning 
tailgates  before  personnel  arrive  on  the  job.  Of  course,  well-described  job  competencies 
reinforced  by  definitive  performance  expectations  ensure  that  personnel  are  properly  placed 
and  appropriately  guided.  Nonetheless,  systems  engineers  should  be  ready  to  apply 
engineering  basics;  contract  specialists  should  be  ready  to  carefully  evaluate  written 
agreements;  and  cost  estimators  should  be  steeped  enough  in  math  to  comfortably  work 
with  budget  and  cost  estimate  equations.  Despite  the  profession,  these  formal  foundational 
learning  gates  are  less  than  half  of  the  total  learning  equation  in  the  workplace.  The 
remainder  occurs  at  the  workplace.  In  fact,  more  than  70%  of  most  new  knowledge  and 
skills  actually  take  place  at  work  through  a  combination  of  informal  and  incidental  learning 
(Good  &  Brophy,  1990).  This  is  where  the  workforce  tests  their  inherent  capabilities  every 
day.  Where  do  experience  catalysts  play  into  all  of  this?  They  appear  to  take  root  more  in 
the  context  of  these  informal  and  incidental  learning  methods  (i.e.,  “learning  by  doing”).  If 
that  is  the  case,  what  did  the  DoD  acquisition  workforce  actually  say  about  the  effectiveness 
and  value  of  these  experience  catalysts  early  on  ‘in”  the  job?  What  mattered  most? 

The  survey  respondents  were  asked  to  rate  the  importance  of  a  broad  range  of 
experience  factors.  The  majority  have  been  well-documented  by  research.  Others,  like  off¬ 
sites  and  immersion  days,  have  not  been  well  documented,  but  for  purposes  of  this 
investigation,  they  have  been  sub-categorized  as  knowledge-sharing  components. 

As  Figure  1  indicates,  the  results  were  consistent  with  previous  research.  On-the-job 
training  mattered  the  most.  Active  involvement  in  the  experience  strengthened  their 
experience  foundations  according  to  many  respondents  in  this  study.  Knowledge  sharing 
with  colleagues  and  challenging  work  trailed  very  closely  behind.  Several  respondents  felt 
“learning  from  others’  experiences  reinforced  their  own.”  Knowledge  sharing  can  have  far- 
reaching  considerations  since  knowledge  is  generally  seen  as  “the  most  strategically- 
important  resource  which  organizations  possess  and  a  principal  source  of  value  creation” 
(Cummings,  2003).  Supervisory  guidance  represented  the  next  data  point.  One  of  the 
respondents  echoed  the  views  of  others.  She  claimed  that  “having  a  well  trained  supervisor 
who  is  a  great  teacher,  allowing  me  to  fly  semi-alone... built  [my]  confidence,  knowledge  and 
courage  to  complete  more  challenging  tasks.”  The  next  lower  grouping  included  DAWIA 
classroom  training,  formal  mentorship,  professional  development,  well-defined  organization 
processes,  on-line  training,  and  certification  standards.  Unexpectedly,  three  of  these  seven 
(DAWIA  training,  well-defined  organizational  processes,  and  certification  standards)  all 
scored  noticeably  low  and  could  be  explained  for  several  reasons. 
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Figure  1.  T1  Experience  Catalysts 

Why  the  low  score  for  DAWIA  classroom  training ? 

■  Its  value  could  be  muted  compared  to  other  more  dominant  experience 
catalysts.  Some  respondents  felt  classroom  experience  will  “never  be  able  to 
replace  of  OJT,  mentoring  or  knowledge  sharing  at  work.”  Others 
emphasized  that  DAWIA  training  is  “rather  generic  and  doesn't  actually  teach 
enough  of  the  job  specifics.”  In  other  words,  the  training  could  be  too  general 
in  nature. 

■  Students  might  be  showing  up  too  early  for  training  in  their  career  and  may 
not  be  quite  ready. 

■  Students  might  be  showing  up  too  late  for  training.  Several  respondents 
noted  that  it’s  difficult  to  keep  up  with  additional  training  demands. 

■  Students  forgot  what  they  learned  before  they  could  apply  it. 

■  It  could  have  a  looser  connection  to  experience  in  its  current  form. 

■  Its  benefits  might  not  be  well  understood,  especially  the  connection  to 
performance  outcomes — something  the  General  Accountability  Office  (GAO) 
recently  questioned. 

In  a  recent  report,  the  GAO  declared  that  without  appropriate  outcome  metrics, 
acquisition  Technology  &  Logistics  programs  will  be  “unable  to  demonstrate  how  certification 
training  actually  contributes  to  organizational  performance  results”  (GAO,  2010). 

Inarguably,  what  the  GAO  underscored  is  tough  to  demonstrate  without  a  comprehensive 
program  that  tracks  behavioral  changes  at  work.  The  discovery  that  as  much  as  90%  of 
training  resources  are  spent  on  the  design,  development,  and  delivery  of  training  events 
only  yield  15%  on-the-job  application  (Brinkerhoff,  2006)  makes  training  an  easy  target  for 
additional  examination.  In  the  context  of  Donald  Kirkpatrick’s  well-  known  Four  Levels  of 
Learning  Evaluation,  the  first  two  learning  levels  (Reaction  [I]  and  Learning  [II])  have  been 
relatively  easy  to  demonstrate  during  the  classroom  delivery  timeframe.  Level  III  (Behavior) 
and  Level  IV  (Results)  have  been  a  lot  tougher  to  validate.  Some  researchers  assert  that  if 
Level  3  evaluations  were  conducted  as  part  of  existing  career  development  and 
performance  reviews,  then  it  might  “improve,  explain,  control,  and  predict  performance 
although  managers  must  be  willing  to  observe,  document,  and  evaluate  the  desired 
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behaviors”  (Mayberry,  2005).  Brinkerhoff  and  Montesino  (1995)  found  even  modest 
supervisor  involvement  before  and  after  the  training  can  have  a  significant  impact  on 
whether  trainees  use  their  newly  developed  skills”  (Bassi  &  Russ-Eft,  1997).  Other  studies 
have  shown  that  “the  more  managers  are  trained  in  how  to  support  and  coach  the  skills  their 
employees  learn,  the  more  those  skills  will  be  used  and  sustained  in  the  workplace” 
(Leimbach  &  Maringka,  2009). 

Decades  ago,  the  DoD  instituted  a  formal  performance  evaluation  program  for  all  its 
employees  to  signal  the  importance  of  training.  In  1958,  the  Government  Employees’ 
Training  Act  expected  training  would  improve  performance  and  prepare  personnel  for  future 
advancement.  In  1962,  the  Salary  Reform  Act  required  an  “acceptable  level  of  competence” 
determination  for  granting  General  Schedule  within-grade  increases;  provided  for  the  denial 
of  the  within-grade  increase  when  performance  is  below  the  acceptable  level;  and 
authorized  an  additional  step  increase  for  “high  quality  performance.”  While  these  formal 
evaluation  measures  have  continued  to  evolve,  they  have  not  specifically  traced  personnel 
performance  to  training  activities.  It  has  been  generally  assumed  that  training  focuses  on 
the  required  knowledge,  skills,  and  abilities  necessary  to  perform  and  improve  assigned 
duties  within  the  workplace.  In  fact,  there  is  plenty  of  literature  that  substantiates  this 
probabilistic  connection,  but  there  are  so  many  other  intervening  factors  that  complicate  the 
relationship,  including  individual  attitude,  motivation,  cultural  realities,  learning  self-efficacy, 
age,  etc.,  that  make  a  deterministic  forecast  more  difficult  (Bassi  &  Russ-Eft,  1997).  Other 
factors  ebb  and  flow,  such  as  team  structures,  incentives,  use  of  analytic  tools  for  capturing 
and  analyzing  information,  and  psychological  safety,  and  tend  to  moderate  the  influence 
between  experience  and  performance  improvement  (Edmondson,  1999).  In  the  private 
business  sector,  training  has  been  found  to  have  a  positive  impact  on  profitability  (Cosh  & 
Hughes,  2003).  Many  years  ago,  the  DoD  made  a  similar  association  for  its  acquisition 
workforce  and  invested  heavily  in  training.  It  still  takes  training  very  seriously. 

As  far  as  other  experience  foundation  catalysts  go,  there  are  several  others  that 
require  further  introspection. 

Why  the  low  score  for  well-defined  organizational  processes? 

■  Personnel  might  believe  these  processes  are  already  embedded  in  the 
direction  they  received  and  might  not  necessarily  see  them  as  a  distinctive 
element. 

■  Personnel  might  be  more  sharply  focused  on  their  day-to-day  tasks  at  hand 
and  not  find  them  a  necessity  (yet). 

■  Personnel  haven’t  found  the  ones  in  place  to  represent  much  value. 

Why  the  low  score  for  certification  standards? 

■  They  could  be  generally  misconstrued. 

■  They  don’t  go  far  enough  and/or  are  too  watered  down  to  be  significant. 

■  The  connection  to  accountability  might  not  be  readily  apparent. 

Why  the  low  score  for  communities  of  practice,  another  form  of  knowledge  sharing? 

■  Personnel  may  not  find  it  a  rich  source  of  useful  knowledge. 

■  Personnel  may  not  find  the  information  current  enough. 

■  Personnel  may  not  be  aware  of  its  existence. 

■  Information  may  not  be  appropriately  curated  (e.g.,  information  has  not  been 
properly  maintained  or  trusted  for  use). 

■  It’s  missing  the  social  interaction  that  generally  creates  more  value. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  274 


2nd  Tier:  Experience  Enhancers 

The  impact  of  experience  catalysts  expressed  as  experience  enhancers  seemed 
relatively  consistent  to  those  described  as  foundational  (Figure  2)  and  were  very  closely 
correlated.  On-the-job  training  didn’t  diminish  in  importance;  neither  did  knowledge  sharing, 
challenging  work,  nor  supervisory  guidance.  In  relative  terms,  they  all  rose  slightly.  While 
still  having  noticeably  fewer  experience  catalysts  than  the  big  three,  both  classroom  training 
and  on-line  training  rose  much  more  noticeably  in  relative  terms. 


Figure  2.  T1  and  T2  Experience  Catalysts 

The  uncharacteristic  rise  in  on-line  training  could  be  attributed  to  (a)  how  on-line 
training  complements  certain  experience  foundations  previously  forged  or  (b)  the  presence 
of  more  effective  delivery  methods  (e.g.,  greater  interactive  features  and  less  of  a  “page 
turner,”  perhaps).  Traditionally,  DAWIA  classroom  training  that  uses  scenario-based 
learning  (SBL)  methods  enjoys  more  of  an  advantage  than  other  classroom  methodologies. 
It  gives  students  a  chance  to  practice  representative  training  scenarios  alongside  their  peers 
and  to  reflect  about  their  jobs  while  they  are  away  from  their  jobs.  Reflection  and  practice 
have  been  found  to  have  a  significant  impact  on  experiential  learning  of  this  kind.  Long  ago, 
David  Kolb,  an  American  educational  theorist,  reported  that  in  order  to  gain  genuine 
knowledge  from  an  experience,  the  learner  “must  be  able  to  reflect  on  the  experience  as 
well  as  be  willing  to  get  actively  involved  in  the  experience;  possess  and  use  analytical  skills 
to  conceptualize  the  experience;  and  possess  decision  making  and  problem  solving  skills  in 
order  to  use  the  new  ideas  gained  from  the  experience”  (Kolb,  1983).  Classroom  training 
that  employs  SBL  does  just  that  and  is  used  extensively  these  days  since  it  adheres  to  a 
performance  improvement  imperative  rather  than  the  acquisition  of  just  knowledge  and  skills 
(Schulz,  2001).  SBL  also  promotes  defining  moments  by  exposing  an  individual’s  strengths 
and  weaknesses.  By  imitating  something  real,  SBL  has  shown  to  pay  huge  experience 
dividends  by  igniting  the  senses.  Many  have  already  found  their  way  into  organizations  that 
vitally  depend  on  training.  Soaked  with  real-world  conditions,  they  test  an  individual’s  ability 
to  demonstrate  how  certain  critical  competencies  prevail  (or  not).  Captain  Chesley  “Sully” 
Sullenberger  III,  a  former  U.S.  Airways  seasoned  pilot,  experienced  the  benefits  first-hand. 
He  spent  the  better  part  of  two  full  days  every  six  months  at  the  controls  of  an  Airbus  319 
flight  SBL  simulator  while  several  lifetimes’  worth  of  disasters  broke  loose  around  him 


np s] 


M  K  ■M.  N.H.  rriHi 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  275 


(Budiansky,  2009).  At  what  point  was  he  prepared  for  a  water  landing  on  the  Hudson  River 
when  he  piloted  Flight  Number  1549  on  January  15,  2009?  How  many  years  did  it  take  for 
him  to  turn  a  potential  disaster  into  a  miracle?  He  met  his  flying  experience  markers  (in 
years),  but  up  to  the  moment  before  he  set  his  aircraft  on  the  Hudson,  a  SBL  simulator 
allowed  him  to  fly  at  the  edge  of  the  flight  envelope  and  test  him  for  just  about  any 
contingency — except  a  water  landing.  The  Airbus  319  isn’t  a  watercraft,  but  Captain  Sully 
knew  he  had  to  treat  it  like  one,  given  the  threatening  outcome  of  two  failed  engines.  His 
many  years  as  an  experienced  line  pilot,  combined  with  recurring  scenario-based  simulator 
training  and  with  the  ability  to  handle  “the  unexpected,”  helped  him  save  155  lives  that  day. 
Aside  from  their  long-standing  presence  in  the  flying  community,  as  long  as  the  experience 
is  seen  as  realistic  and  valid,  simulators  also  show  promise  for  many  other  professions  that 
require  continuous  practice  and  steady  reinforcement. 

Virtual  simulators  were  previously  an  expensive  proposition.  Not  anymore.  Now, 
high-fidelity  virtual  simulations  and  the  introduction  of  gaming  that  even  uses  3-D  capability 
are  relatively  inexpensive  and  widespread.  They  could  eventually  become  commonplace  in 
many  workplace  settings.  When  that  occurs,  they  might  have  a  noticeable  impact  on 
experience  gains  for  many  professions  by  letting  workers  safely  practice  a  wide  range  of 
challenges  unique  to  their  own  areas  of  expertise  on  the  job. 

3rd  Tier:  Experience  Accelerators 

The  data  associated  with  this  last  tier  resulted  in  several  interesting  surprises.  First, 
there  were  fewer  correlations  with  1st  and  2nd  Tier  factors.  Second,  professional 
development,  well-defined  organizational  processes,  and  formal  mentorship  took  a  marked 
leap  in  importance  as  accelerators  (Figure  3).  Third,  challenging  work  and  certification 
standards  took  visible  dips.  What  caused  certain  experience  catalysts  to  rise  in  importance 
and  others  to  fall? 


Figure  3.  T1,  T2,  and  T3  Experience  Catalysts 

The  following  could  help  explain  the  T3  experience  factors  that  rose  in  importance: 

■  The  rise  in  professional  development  (i.e.,  off-the-job  training)  could  be 
attributed  to  the  potential  knowledge  gains  found  outside  the  workplace  on 
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supplementary/complementary  subjects  and/or  interactive  knowledge-sharing 
venues  with  leaders  in  their  same  fields.  Some  respondents  emphasized  the 
importance  of  training  with  industry  and  other  professional  development 
opportunities. 

■  The  rise  in  well-defined  organizational  processes  could  be  attributed  to  the 
tangible  benefits  of  more  definitive,  written  organizational  guidance  that  might 
have  been  less  obvious  before.  Research  has  shown  that  learning  from 
direct  experiences  depends  critically  on  organizational  processes  that 
generate  experiences.  (Schultz,  2001) 

■  The  rise  in  formal  mentorship  could  be  attributed  to  personnel  seeking  advice 
and  counsel  from  more  seasoned  professionals  in  their  same  career  fields  in 
their  own  work  environment.  One  respondent  commented  that  “having  a 
hands-on  mentor  made  a  world  of  difference.”  Another  stated  that  “having  a 
hands-on  mentor  at  the  start  of  their  career  would  have  made  a  world  of 
difference.” 

A  few  possibilities  could  help  explain  the  T3  experience  factors  that  dropped  in 


importance. 


■  The  dip  in  challenging  work  could  be  attributed  to  the  following: 

o  The  work  at  hand  may  no  longer  be  challenging  enough  and  could  be 
holding  people  back. 

o  Good  work  is  rewarded  with  more  work  and  eventually  could  feel  more 
like  work  overload  without  the  time  to  adequately  learn  it. 
o  The  complicating  effects  of  increased  administrative  burden  (seen  by 
some  as  more  work)  is  too  much  sidebar  work  to  promote  any  real 
experience  gains. 

■  The  dip  in  certification  standards  (and  the  lowest  of  all  experience 
accelerators)  could  be  attributed  to  the  following: 

o  Poorly  described  benefits — some  personnel  may  not  easily  see  the  future 
professional  and  personal  payback. 

o  Personnel  may  also  find  the  achievement  thresholds  too  low  or  less 
relevant  to  their  current  jobs,  as  many  of  the  respondents  stated, 
o  The  time  they  were  awarded  their  certification  level  was  long  ago. 


Barriers 


Over  time,  experience  undeniably  prepares  the  workforce  for  the  challenges  ahead. 
However,  past  experience  can  also  artificially  interfere  with  the  need  for  innovation  and 
modernization,  something  the  DAW  or  any  other  profession  can  least  afford. 

Epistemologists  (who  study  the  theory  of  knowledge)  might  argue  that  knowledge,  skills,  and 
attitudes  (KSAs)  are  so  tightly  connected  to  experience  that  they  could  become  a  little  too 
grounded  in  yesterday’s  beliefs  and  dismiss  the  truths  that  might  no  longer  apply.  In  other 
words,  the  same  attributes  that  yield  conventional  wisdom  could  sometimes  produce  fixed 
mindsets,  superstitious  learning  (e.g.,  single  perspectives,  learning  the  wrong  things), 
competency  traps,  and  erroneous  inferences  (Levitt  &  March,  1988).  Before  1947, 
engineers  believed  the  speed  of  sound  represented  a  physical  barrier  for  aircraft  (and  pilots) 
because  of  the  formation  of  a  violent  shock  wave  that  would  dramatically  increase  drag, 
induce  uncontrollable  shaking,  lose  airlift,  and  eventually  cause  complete  flight  control 
failure.  Placing  a  man  on  the  Moon  then  must  have  been  an  absolutely  wild  idea.  That  all 
changed  when  Chuck  Yeager  broke  the  sound  barrier  in  the  Bell  X-1  “Glamorous  Glennis” 
on  October  14,  1947;  and  when  Neil  Armstrong  walked  on  the  Moon  on  July  20,  1969. 
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As  previously  indicated,  this  research  study  also  looked  at  the  presence  of  barriers 
that  could  be  interfering  with  experience  gains.  The  respondents  were  asked  if  the  lack  of  or 
reduction  in  these  experience  catalysts  served  as  barriers.  As  Figure  4  shows,  the  barriers 
followed  a  close  inverse  correlation  to  experience  accelerators.  These  barriers  did  not 
necessarily  predominate,  but  they  did  highlight  a  couple  of  areas  worth  further  examination. 


Figure  4.  T1,  T2,  and  T3  Experience  Catalysts  &  Barriers 

The  lack  of  well-defined  organization  processes  (also  seen  as  an  experience 
accelerator  when  visibly  present)  were  the  most  prominent  and  could  be  attributed  to 

■  outdated  processes  that  no  longer  apply; 

■  reduced  support  for  existing  organizational  processes; 

■  ambiguity  that  certain  key  organizational  processes  even  exist;  and 

■  poorly  conveyed  guidance  without  adequate  explanation  or  appropriate 
justification  (One  respondent  stated  that  the  lack  of  published  work 
processes  curbed  his  experience  gains). 

While  less  of  a  barrier,  although  still  noticeable,  the  lack  of  formal  mentorship  (and 
also  seen  as  an  experience  accelerator  when  visibly  present)  emerged  as  a  barrier, 
suggesting  that  some  personnel  might  need  more  coaching  from  more  senior  personnel 
whom  they  trust  and  respect. 

The  lack  of  participation  in  communities  of  practices  (CoPs)  was  considered  neither 
a  barrier  nor  a  substantial  experience  factor  in  any  one  of  the  three  tiers.  While  CoPs  can 
give  access  to  a  tremendous  set  of  colleagues  steeped  in  relevant  knowledge  and 
experience,  they  appear  to  have  less  of  an  impact  on  experience  growth  than  expected. 

Certification  standards  were  not  seen  as  a  barrier,  suggesting  that  the  workforce  did 
not  necessarily  see  them  as  either  inhibiting  experience  gains  or  helping  to  achieve  them. 

Recommendations 

The  data  in  this  study  confirmed  the  substantial  influence  of  certain  experience 
catalysts  where  they  tend  to  predominate — in  the  workplace.  Understanding  the  correlation 
and  value  of  these  high  flyers  can  have  a  marked  impact  on  individual  performance  and 
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acquisition  outcomes  if  fully  exploited.  The  experience  catalysts  operating  in  a  less 
influential  state  could  perhaps  have  a  noticeable  impact  as  well.  If  appropriately  recognized 
(and  in  some  cases,  either  clarified  or  [re]energized),  they  could  also  represent  a  powerful 
force  multiplier  for  even  more  experience  gains. 

The  acquisition  workforce  who  participated  in  this  study  re-validated  the  major 
experience  gains  achieved  by  work-related  experience  catalysts.  Accordingly,  the  sooner 
that  formal  training  and  informal  training  converge,  the  greater  the  impact  off-the-job  training 
will  have.  More  importantly,  however,  is  the  convergence  between  on-the-job  and  off-the- 
job  experience  catalysts.  When  these  two  converge,  they  will  better  prepare  the  workforce 
for  many  more  on-the-job  eventualities  (Figure  5). 


Figure  5.  Convergence  of  Off-the-Job  and  On-the-Job  Experiences 

Convincing  organizations  to  fully  grasp  the  actuality  that  they  also  serve  as  informal 
learning  organizations  where  experience  really  takes  root  (i.e.,  on  the  job)  could  serve  as  a 
crucible  for  many  experience  catalysts.  To  meet  that  end,  the  following  recommendations 
are  warranted  for  defense  acquisition  operating  units: 

1 .  Codify  acquisition  operating  units  as  learning  organizations.  Recognize  the 
wide  range  of  experience  catalysts  in  use  daily  in  the  workplace  and  how  they 
can  favorably  impact  organizational  outcomes.  Institute  and  monitor  with 
regular  frequency  the  effect  of  these  experience  catalysts  inside  the 
organization,  and  adjust  as  required.  Reduce  the  barriers  that  might  be 
limiting  certain  experience  gains.  More  specifically, 

a.  Keep  the  work  challenging  and  in  perspective.  The  acquisition 
workforce  expects  to  be  challenged — a  key  part  to  their  professional 
growth.  Workers  tend  to  stay  at  the  job  and  keep  focused  when  the 
work  is  challenging  and  relevant.  They  leave  when  the  work  is  not. 
Even  when  the  tasks  are  sometimes  repetitious,  one  respondent 
commented  that  they  were  different  enough  to  strongly  influence  his 
motivation  to  stay  since  he  also  saw  the  impact  he  was  having. 

b.  Capitalize  and  promote  knowledge  sharing  opportunities.  Build  an 
organic,  flexible,  logical,  current,  and  enduring  information 
architecture  warehouse  that  contains  actionable  information  that 
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personnel  can  tap  into  freely.  Give  personnel  easy  access  to  key 
information  sources  of  expertise.  It  deepens  the  workforce’s 
knowledge  base,  expands  perspectives,  and  fuels  their  experience 
engine.  Without  the  open  and  continuous  dialogue  with  peers, 
colleagues,  and  experts,  competency  gaps  are  more  likely  to  occur 
and  experience  growth  might  plateau.  Give  project  teams  enough 
time  to  process  new  information.  Personnel  need  slack  time, 
organizational  experience,  and  decision-making  autonomy  to  fully 
benefit  from  access  to  new  knowledge  (Haas,  2006).  Reward 
personnel  for  integrating  and  applying  new  knowledge  when  it  creates 
organizational  performance  gains. 

c.  Get  supervisors  involved  in  the  training  process  before  and  after  the 
event.  With  greater  involvement  by  the  supervisor,  the  training  can 
have  more  relevance  and  create  more  favorable  impacts  back  on  the 
job.  The  most  important  work  environment  factors  affecting  training 
transfer  include  “discussions  with  the  supervisor  on  the  use  of  new 
learning,  the  supervisor’s  involvement  or  familiarity  with  the  training 
and  positive  feedback  from  the  supervisor”  (Nijman,  201 1 ).  Just  as 
importantly,  supervisors  need  to  measure  the  performance  outcomes 
of  their  personnel  back  on  the  job  and  show  the  dividends  that  training 
has  produced.  If  the  training  is  not  hitting  the  mark,  the  first  line 
supervisors  would  be  the  first  to  know  and  should  clarify  what  needs 
to  change  through  the  appropriate  channels.  Supervisor  commitment 
is  crucial  in  validating  the  usefulness  of  training. 

d .  Clearly  articulate  and  punctuate  the  effectiveness  of  organizational 
processes.  Keep  processes  current,  effective,  and  relevant. 
Communicate  their  usefulness  with  regular  frequency.  Show  the 
benefits.  Consider  revising  or  terminating  the  ones  that  have  outlived 
their  usefulness.  Ask  the  workforce  what  needs  to  change  (or  not). 
They  are  just  as  much  the  owners  of  the  process  as  anyone  else. 

e.  Promote  and  support  professional  development  opportunities.  It 
broadens  the  workforce’s  knowledge  base  by  giving  them  an 
opportunity  to  further  develop  themselves  by  reaping  the  experiences 
and  effective  practices  of  others.  It  creates  new  professional 
relationships  and  future  experience  networks  that  the  workforce  can 
leverage  for  years  to  come.  It  makes  an  organization  stronger.  It 
combats  competency  gaps  and  helps  break  down  obsolete  mental 
mindsets. 

f.  Promote  mentorship.  Recipients  can  draw  from  the  rich  experiences 
from  seasoned  and  respected  leaders  who  possess  a  wealth  of 
experience.  Mentors  encourage  introspection.  They  motivate  and 
inspire.  They  can  help  build  a  sustainable  career  pathway  for 
personnel  who  are  looking  to  widen  their  experience  gains  in  both 
depth  and  breadth  as  they  pursue  their  professional  careers. 

g.  Recognize  the  efficacy  of  DAW! A  training.  Ensure  that  workforce 
members  are  ready  for  the  training  and  the  training  is  meeting  the 
needs  of  the  workforce.  Provide  useful  and  timely  feedback  to  the 
training  communities. 
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h.  Explore  how  immersion  days  and  off-sites  could  promote  and  result  in 
experience  gains  for  personnel  back  on  the  job.  These  very  focused 
events  cover  a  lot  of  ground,  but  the  agenda  should  also  include  a 
component  that  targets  individual  and  organizational  performance.  If 
they  don’t,  then  their  connection  to  experience  catalysts  will  remain 
nominal. 

The  following  recommendations  are  warranted  for  defense  acquisition  training 
organizations: 

2.  Tighten  the  connection  between  off-the  job  training  and  on-the-job  training. 
Learners  need  to  understand  the  connection  by  witnessing  the  connection. 
The  clearer  the  link  between  the  skills  taught  and  the  skills  required  at  work, 
the  more  required  and  newly  acquired  skills  will  stick.  Make  it  truly 
experiential.  Validate  the  learning  objectives  taught  in  class  with  outcomes  in 
the  field  through  a  pain-free  measurable  follow-up  initiative  later  in  the  field. 
Specialize  and  personalize  the  training  whenever  possible.  Mimic  their  work 
environments  as  much  as  possible  through  methods  that  truly  ignite  their 
senses.  In  his  book,  Talent  is  Overrated,  Geoff  Colvin  argues  that  “the  road  to 
exceptional  performance  is  the  result  of  deliberate  practice”  (Colvin,  2010)  at 
what  they  need  to  know  how  to  do. 

a.  Maximize  SBL.  Few  training  techniques  mimic  the  actual  work 
environment  better.  They  test  the  workforce  under  realistic  conditions 
and  give  the  workforce  a  chance  to  show  their  grit  without  the  threat  of 
dangerous  consequences.  It  also  brings  together  both  cognitive  (e.g., 
mental  processes,  knowledge  application)  and  affective  (e.g.,  feelings, 
attitude)  behaviors,  thereby  increasing  the  quality  of  the  experience. 
“Everything  depends  on  the  quality  of  the  training  experience” 

(Dewey,  1998). 

b.  Reinforce  the  benefits  of  certification  standards.  Show  the  workforce 
the  proof.  While  it  should  have  bearing  on  upward  mobility,  it  should 
not  be  the  principal  motivator.  Too  many  acquisition  professionals  still 
do  not  see  the  returns.  Many  respondents  saw  getting  their 
certifications  as  a  way  to  get  promoted  and  represented  some  of  their 
motivation  to  take  the  course  in  the  first  place. 

c.  Monitor  the  usefulness  of  communities  of  practices  (CoPs)  closely. 
Either  reinvigorate  certain  CoPs  that  have  dropped  sharply  in 
popularity  or  replace  them  with  other  knowledge-sharing  methods  that 
show  more  promise.  If  seen  as  invaluable,  personnel  will  frequent. 
CoPs  can  provide  the  workforce  with  tremendous  access  to  a  wider 
experience  network,  but  it  has  to  go  beyond  simple  data  transmission. 
Research  evidence  shows  that  knowledge-sharing  methodologies 
involving  people  interactions  are  superior  to  those  involving  only 
document  exchanges  since  knowledge  often  needs  to  be  carefully 
adapted  to  a  new  context  in  order  for  it  to  be  effectively  utilized 
(Cummings,  2003). 


Conclusion 

Today,  in  the  face  of  declining  budgets  and  increased  public  scrutiny  of  every  dollar 
the  DoD  spends,  the  defense  acquisition  workforce  is  facing  growing  pressure  to  make 
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every  dollar  for  its  goods  and  services  count.  While  experience  has  and  will  continue  to  be 
a  fundamental  component  of  the  human  capital  development  equation,  it  is  vitally  important 
that  the  DoD  recognizes  what  experience  catalysts  matter  the  most  to  the  acquisition 
workforce.  Twenty  years  from  now,  experience  inside  the  DAW  will  matter  just  as  much  as  it 
did  when  Congress  voted  the  Defense  Acquisition  Workforce  Improvement  Act  into  law  over 
20  years  ago.  The  only  difference  might  be  that  the  seam  between  off-the-job  training  and 
on-the-job  training  will  disappear.  The  warfighting  community  has  already  reaped  the 
benefits  of  a  similar  transparency.  Like  few  others,  warfighters  test  their  experience  in 
warfighting  exercises  that  mimic  real-world  conditions  under  fire.  The  experience  they 
possess  grew  from  what  mattered  most:  They  train  like  they  fight  and  fight  like  they  train. 
When  the  acquisition  community  at  large  is  tested  in  a  similar  fashion  through  intellectual 
workouts  that  mimic  their  real-world  conditions,  performance  outcomes  will  invariably  rise. 

The  acquisition  workforce  would  be  well  served  if  it  recognizes  the  importance  and 
influence  of  all  experience  catalysts  operating  in  the  upper  bands  and  better  leverages  the 
confluence  of  them — even  the  ones  operating  in  the  lower  bands.  Granted,  there  are  so 
many  variables  involved  in  the  experience  equation.  However,  the  key  in  its  application 
depends  on  whether  the  workforce 


■  continuously  practices  their  craft  at  work  in  what  has  long  been  serving  as  on- 
the-job  laboratories; 

■  applies  their  mettle  with  challenging  work  and  supervisors  close  by,  and  with 
mentors  not  far  away; 

■  consistently  shares  relevant  information  through  a  highly  collaborative  and 
open  knowledge-sharing  environment  in  a  wide  range  of  mediums; 

■  recognizes  the  necessity  and  compelling  reason  for  and  connection  between 
training  and  certification;  and 

■  continuously  thinks  beyond  yesterday’s  truths  without  getting  trapped  by 
competency  gaps  that  could  prevent  experimentation  with  more  suitable 
alternatives. 


Implementing  these  collective  actions  might  just  energize  many  of  the  experience 
catalysts  enough  to  the  point  that  they  all  start  to  behave  as  experience  accelerators. 
“Experience  is  the  name  every  one  gives  to  their  mistakes”  (Wilde,  1892).  The  DoD’s 
acquisition  workforce  can  least  afford  any  experience  shortfall  that  results  in  weapon  system 
delays  for  warfighters  serving  in  harm’s  way.  Warfighters  depend  on  the  DAW  to  get  it  right 
the  first  time,  and  that’s  the  only  “aha”  that  really  matters. 
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Tl:  Foundational 


•  Tl:  FOUNDATIONAL 
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Tl(Foundational),  T2  (Enhancers), 
T3  (Accelerators),  Barriers 


•  Tl:  FOUNDATIONAL  -#-12:  ENHANCERS  -*-T3:  ACCELERATORS  -B-BARRIERS 


Recommendations 


Recommendations  for  Acquisition 
Business  Units 

•  Codify  yourselves  as  Learning  Organizations 

•  Keep  the  work  challenging  and  in  perspective 

•  Capitalize  and  promote  Knowledge  Sharing 

•  Get  supervisors  involved  in  the  training  process  before  and  after 
the  event 

•  Clearly  articulate  and  punctuate  the  effectiveness  of 
organizational  processes 

•  Promote  and  support  professional  development 

•  Promote  mentorship 

•  Recognize  the  efficacy  of  DAWIA  training 

•  Explore  benefits  of  Immersion  Days  and  Off-sites 

•  Leverage  social  media 


Recommendations 


Recommendations  for 
Acquisition  Training  Units 


Continue  to  tighten  the  connection 
between  off-the  job  training  and  on-the-job  training 

•  „  "To  Be" 

As  is 
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Recommendations 


Recommendations  for 
Acquisition  Training  Units 

*  Maximize  Scenario  Based  Learning 

*  Reinforce  the  benefits  of  certification  standards 

*  Monitor  and  exploit  the  usefulness  of  all  Knowledge 
Sharing  mediums  including  social  media 
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Conclusion 


Experience  is  the  name  every  one  gives 
to  their  mistakes"  (Oscar  Wilde,  1892) 

*  The  DoD's  acquisition  workforce  needs  the 
time  to  practice  just  like  any  other  profession  and  can 
least  afford  any  experience  shortfall  that  results  in 
weapon  system  delays  for  warfighters  serving  in 
harm's  way. 

*  Warfighters  depend  on  the  DAW  to  get  it  right  the 
first  time 


